Filter stack and radial covar optimizations by garrettwrong · Pull Request #1388 · ComputationalCryoEM/ASPIRE-Python

garrettwrong · 2026-06-11T18:16:11Z

Converts code towards filter stacks while maintaining ability to use legacy single filters (for now).

Provides a to_radial method for Filters.

Provides means to mix most singleton and stack filters. Vectorizes bulk evaluations.

Tries to vectorize (and sometimes use GPU when things fit) for filter_statck_to_basis_mats.

Add code paths to run radially optimized filter_stack_to_basis_mat.

Adds a smoke/timing test for covar2d using 10028 (ribsome) and 11618 (ctf per image) experimental sized cases.

Still a WIP, need self review.

codecov · 2026-06-11T18:36:17Z

Codecov Report

❌ Patch coverage is 97.03504% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.65%. Comparing base (20a8ac2) to head (39bcdb4).

Files with missing lines	Patch %	Lines
src/aspire/operators/filters.py	96.02%	6 Missing ⚠️
src/aspire/basis/fle_2d.py	93.02%	3 Missing ⚠️
src/aspire/basis/ffb_2d.py	95.65%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1388      +/-   ##
===========================================
+ Coverage    90.56%   90.65%   +0.08%     
===========================================
  Files          135      135              
  Lines        14691    14926     +235     
===========================================
+ Hits         13305    13531     +226     
- Misses        1386     1395       +9

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[skip ci]

warning sysetm needs improving

still needs cleanup

…al filters

garrettwrong · 2026-07-08T13:57:15Z

Going to kick off the full CI suite.

janden

Nice work! A few questions here and there, but nothing major.

janden · 2026-07-09T12:47:32Z

+        """
+        Expands radial vector or stack of vetors `radial_vec` to basis matrix.
+
+        :param radial_vec: Array holding radial vector,


But how do we know whether these radial pts are compatible with the radial grid used here (from _precomp["gl_nodes"])?

For the radial case, gl_nodes did not work well as-is, I had to remake them in double precision :/

# Weights appear a little sensitive to dtype, otherwise could use self._precomp["gl_nodes"]

I changed the occurrence of 0.5 to self.kcut in (one new, one existing)lgwt calls. kcut is fixed to 0.5. That would more/less enforce self consistency within the basis.

As far as the input to the function... ASPIRE code is generating the radial vectors by using basis._filter_pts. If that attribute doesn't exist the code I wrote to generate the radial vector will not succeed and the code will not reach this point. Now, If a user is cooking their own radial vector using their own code, that is their responsibility. They can read the code.

I've added

See `_filter_pts` for point set.

to both expand_radial_vec docstrings.

janden · 2026-07-09T12:49:02Z

+            for _ in radial_vec
+        ]
+
+        ind_ell = 0


A lot of the work here seems to be doubling up what's done in filter_stack_to_basis_mats. Would refactoring be an option here?

Yes, it is. The first half of those methods are totally different, but they are similar after that. I'll try seeing what calling expand_radial_vec looks like after the averaging to a 1d vector in _filter_stack_to_basis_mats.

Yeah, that seemed to work out fine. I'll try similar for FLE.

Okay. So I was able to make both FFB and FLE call their own expand_radial_vec right after their 2d to 1d averaging. I had to modify the FLE functions a little to do this, as their expand_radial_vec was doing some additional Fourier padding by default.

janden · 2026-07-09T20:17:10Z

        )

-    def filter(self, filter):
+    def convolve(self, filter_values):


Having both filter and convolve can be confusing since they basically describe the same operation. How about calling this one _filter and calling it from filter as done below?

janden · 2026-07-09T20:18:32Z

+        :param filter: An object of type `Filter`.
+        :return: A new filtered `Image` object.
+        """
+        # Note image and filter data is intentionally migrated via


I think this comment is stale. The xp.asarray migration happens above in convolve. The last sentence is also missing parts.

yeah, my bad, that was sloppy on my part. I must have got distracted. Fixed.

janden · 2026-07-09T20:37:18Z

+        """
+        return 1
+
+    def _ctf_params(self):


Are these still needed? I don't recall seeing them used anywhere.

I originally wrote it as a bridge (before the filter stack conversion). I left it for three minor reasons.

It ended up making a call in Simulation cleaner/simpler. filter_stack_params = self.filter_stack._ctf_params()[ :, :6 ] replaces a lookup loop over starfile mapping/attributes before the data is being repacked from per-filter to per-image for the set-metadata call. (The mapping still exists, its just implicit in the structure now.) Ie, it moves the lookup/packing of the filter attributes from Simulation to Filter code, which is Filter business imo. Then the only thing that happens in Simulation is the ctf to image mapping, which is Simulation business.

If anyone ever "just wants all the ctf params" It's way more convenient one-liner than dropping the starfile (gross) or duplicating code that teases them out one-by-one (ie someone copying the loop code out of from Simulation, which I've had to do several times now. For example...).

I use it to add tests for equality/inequality of {Radial}CTFFilters which is useful for debugging.

I can revert it if non of those seem worth having the code change.

janden · 2026-07-09T20:43:16Z

+
+        return CTFFilter(
+            self.voltage[items],
+            # self.wavelength[items],


sure! missed that.

janden · 2026-07-09T20:47:12Z

        # Additionally we upcast so downstream computations remain in doubles.
+
+        # First prepare arrays for broadcasting.
+        voltage = xp.atleast_1d(voltage)[:, None]


Shouldn't this be np since we haven't moved over to GPU yet?

In the PR I added an agnostic atleast_1d that will return the type of array (np/cp) it is given. If we're in xp as cupy mode, that will be the one used here.

The way it is written, ctf_formula should accept the inputs on host, gpu, or a mix and probably not crash.

Reverting would limit the param inputs to host only. I'd leave it.

garrettwrong · 2026-07-10T17:12:16Z

@janden for the _ctf_params and atleast_1d, just let me know what you'd prefer to see in the code and I'll change it if need be. Then I can have this back to you while I re-run all the tests. (That will take a while unfortunately).

garrettwrong added enhancement New feature or request Optimization Performance or Resource Optimzation GPU labels Jun 11, 2026

garrettwrong force-pushed the rco_filter_stack branch 2 times, most recently from 1102a7e to 5b5ea36 Compare June 22, 2026 18:47

garrettwrong added 24 commits June 23, 2026 11:05

add timing printouts within cov2d

49d17ff

add fetch and eval_t times

822e9f9

stashing initial rad ctf port, trying compare to our filter

66cf925

stashing to_radial and freq pt scaling patches

9680000

cleanup debugging logic a bit

a88fd9b

continue cleanup

f621822

[skip ci]

use existing expand_method, add radial, instead of new flag

c84a96c

use existing expand_method, add radial, instead of new flag

798951b

cleanup

4568e6f

cleanup

4a562e7

remove warnings

45a6caf

warning sysetm needs improving

fix logic error

af522e7

refix logic error

1713e0f

add tqdm to cov2d filter to basis mat

7ab9c37

optimal fle basis comp

780a166

stub in bulk ctf code

af7d1bc

still needs cleanup

tests passing except FFB opts

74e60bf

ffb equiv checkpoint

0a1f3c5

cleanup

0dc76ee

cleanup

0582bf7

cleanup FFB doc strings

faad4ca

dtype sensitivity

1cf187a

more cleanup, maybe CI passing

042bad7

more cleanup, maybe docs runs

338ebfa

increase testing

6c11b8f

garrettwrong force-pushed the rco_filter_stack branch from d141fea to 6c11b8f Compare June 24, 2026 15:53

garrettwrong added 3 commits June 24, 2026 15:02

force covering the batching case

091697d

remove redundant function

df12950

consolidate redundant functions

8b9beee

garrettwrong force-pushed the rco_filter_stack branch from 4b85c73 to 8b9beee Compare June 25, 2026 13:48

garrettwrong added 5 commits June 29, 2026 09:35

comment cleanup

8b8cec9

cleanup filter src and test file

e0f2311

cleanup

466311a

breakout Image convolve from Image filter

a7b0282

minor cleanup new test file

8d9e1ae

garrettwrong force-pushed the rco_filter_stack branch from e062964 to 8d9e1ae Compare June 30, 2026 11:52

garrettwrong added 4 commits July 2, 2026 07:23

add ability to force non-radial filter_stack_to_basis_mats given radi…

82423b2

…al filters

hide/note force_diag

84b5339

Remove incorrect indice conversion (bug) from convolve wts

6b61e26

add additional cases to long covar test suite

527f7af

garrettwrong marked this pull request as ready for review July 8, 2026 13:57

garrettwrong requested a review from janden as a code owner July 8, 2026 13:57

garrettwrong requested a review from j-c-c July 8, 2026 14:36

janden requested changes Jul 9, 2026

View reviewed changes

garrettwrong added 8 commits July 10, 2026 11:14

rename convolve operation to _filter, rm copy pasta comment

76a5c1e

remove deprecated wavelength attr and comment from CTFFilter

6400090

update kcut

4c80909

add comment regarding _filter_pts

a767ea9

refactor filter_stack_to_basis_mats to use expand_radial_vec

f8cf103

cleanup the old filter_stack_to_basis_mats a little more

ffadf43

refactor FLE ftbm to use expand_radial_vec

aaa8702

tox cleanup

39bcdb4

Uh oh!

Conversation

garrettwrong commented Jun 11, 2026

Uh oh!

codecov Bot commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

garrettwrong commented Jul 8, 2026

Uh oh!

janden left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

garrettwrong commented Jul 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov Bot commented Jun 11, 2026 •

edited

Loading